“You Can Always Do Better!” The Impact of Social Proof on Participant Response Bias

Evaluations of technological artifacts in HCI4D contexts are known to suffer from high levels of participant response bias— where participants only provide positive feedback that they think will please the researcher. This paper describes a practical, low-cost intervention that uses the concept of social proof to influence participant response bias and successfully elicit critical feedback from study participants. We subtly exposed participants to feedback that they perceived to be provided by people ‘like them’, and experimentally controlled the tone and content of the feedback to provide either positive, negative, or no social proof. We then measured how participants’ quantitative and qualitative evaluations of an HCI artifact changed based on the feedback to which they were exposed. We conducted two controlled experiments: an online experiment with 245 MTurk workers and a field experiment with 63 women in rural India. Our findings reveal significant differences between participants in the positive, negative, and no social proof conditions, both online and in the field. Participants in the negative condition provided lower ratings and a greater amount of critical feedback, while participants in the positive condition provided higher ratings and a greater amount of positive feedback. Taken together, our findings demonstrate that social proof is a practical and generalizable technique that could be used by HCI researchers to influence participant response bias in a wide range of contexts and domains.